Task-dependent Optimal Weight Combinations for Static Embeddings

A variety of NLP applications use word2vec skip-gram, GloVe, and fastText word embeddings. These models learn two sets embedding vectors, but most practitioners only one them, or alternately an unweighted sum both. This is the first study to systematically explore a range linear combinations between second sets. We evaluate these on set six benchmarks including IR, POS-tagging, sentence similarity. show that default are often suboptimal demonstrate 1.0-8.0% improvements. Notably, GloVe’s its least effective combination across tasks. provide theoretical basis for weighting embeddings more than other according algorithm task. apply our findings improve accuracy in cross-lingual alignment navigational knowledge by up 15.2%.